1. Field of the Invention
The present invention relates to a method of displaying a three-dimensional image, a method of capturing a three-dimensional image, and an apparatus that displays a three-dimensional image.
2. Description of the Related Art
Various systems are known for apparatuses that display three-dimensional images, i.e. stereoscopic images, and that can display motion pictures, that is, three-dimensional displays. In recent years, there has been a growing demand particularly for a system of a flat panel type which does not require the use of specific viewing exclusive glasses or the like. Some three-dimensional motion picture display apparatuses of this kind utilize the principle of holography, which is difficult to put into practical use. In a known system that can display three-dimensional images relatively easily, a parallax barrier (also referred to as a ray control element) is installed immediately before a direct-vision or projection type display apparatus, for example, a display panel (display apparatus) such as a liquid crystal display apparatus or a plasma display apparatus which has fixed pixel positions; the parallax barrier controls light rays from the display panel so that the rays are directed to an observer.
The parallax barrier controls light rays so that different images can be viewed even at the same position on the parallax barrier by varying angle. Specifically, a slit or a lenticular sheet (cylindrical lens array) is used to provide only lateral parallax (horizontal parallax). A pinhole array or a lens array is used to provide vertical parallax in addition to lateral parallax. Systems using the parallax barrier are further classified into a binocular system, a multiview system, a super-multiview system (a multiview system to which super-multiview conditions are added), and an integral imaging system (simply referred to as an II system below). The basic principle of these systems is substantially the same as that of a system invented about ten years earlier and used for three-dimensional photographs.
The integral imaging system (II system) is described using terms for three-dimensional photography. Accordingly, some documents refer to it as integral photography (also referred to as IP below).
The II system is characterized by having a high degree of freedom for a viewpoint position and enabling easy three-dimensional viewing. A display apparatus with a high resolution can be relatively easily implemented using an II system having only horizontal parallax and not having vertical parallax (simply referred to as a one-dimensional II system or 1D-II system). In contrast, the binocular or multiview system provides only a narrow range of viewpoint positions at which the observer can three-dimensionally view images, that is, a narrow viewing area. Thus, disadvantageously, this system fails to allow the observer to see images easily. However, the binocular or multiview system is the simplest configuration for three-dimensional image display apparatuses. This system also has the advantage of allowing display images to be easily created.
In general, the difference between the II system and the lenticular sheet (LS) system is a plane on which pixels are present, that is, an image plane or a focal plane. However, in a practical design, particularly with a large number of pixels, the difference between the image plane and the focal plane is at most 0.1 mm even without any aberration. It is thus difficult to distinguish the image plane from the focal plane owing to a limit on accuracy. It is also difficult to determine whether light rays are converged at a viewing distance, owing to the limit on accuracy. The term II system, as used in the specification, means a system which does not involve the distinction based on the pixel position on the image plane and on the focal plane but which uses an arbitrary (continuous) lateral (substantially horizontal) viewpoint position at which a normal three-dimensional image is visible at the viewing distance. Further, the term multiview system means a system which is not equivalent to the LS system (does not require the convergence of light rays) and with which at the viewing distance, the normal three-dimensional image is visible at a lateral viewpoint position determined on the basis of interocular distance.
With either the II system or the multiview system, the viewing distance is normally finite. Accordingly, a display image is created so that a perspective projection image at that viewing distance is actually visible. With the II system providing only horizontal parallax and not providing vertical parallax, if the horizontal pitch of the parallax barrier is an integral multiple of the horizontal pitch of the pixel, a set of parallel rays is produced (referred to as parallel-ray II below). Further, the parallel-ray II system provides a correctly projected three-dimensional image by displaying on a display surface a parallax interleaved image obtained by dividing a parallax component image into pieces corresponding to respective pixel arrays, the parallax component image having been perspectively projected at a specified viewing distance in the vertical direction and orthographically projected in the horizontal direction, and then appropriately arranging the pieces.
It is difficult to implement an image capture apparatus that photographs images by varying the projection method and projection central distance between the vertical direction and the horizontal direction because in particular, orthographic projection requires a camera or lens having the same size as that of a subject. Accordingly, to obtain orthographic projection data through image capture, it is practical to convert perspective projection image capture data into divided and arranged image data. In addition, for example, a method is known which interpolates image data using EPI (epipolar plane image).
With the II system, as disclosed in J. Opt. Soc. Am. A vol. 15, p. 2059 (1998), the front- and rear-side reproducible ranges of a three-dimensional image display apparatus are relatively narrow; typical values are about 10 cm in each direction. However, to eliminate restrictions on the position of an object to be displayed, it is necessary to increase the degree of freedom in the front- and rear-side reproducible ranges of the three-dimensional image display apparatus. Some multiview systems that synthesize perspective projection images to form a three-dimensional image display perspective-projection three-dimensional images in which the front and rear sides are emphasized (three-dimensional images which are compressed in a rear-side direction and which have nulls close to the camera). However, with the parallel-ray II system that forms orthographic projection images into a three-dimensional image, no three-dimensional images such as those described above have been created and no methods for creating such images have been found.
The parallel-ray II system advantageously provides easier-to-see images than the binocular system. However, the parallel-ray II system uses a complicated image format in terms of the projection method or division and arrangement method. The binocular and multiview system performs the simplest display of three-dimensional images and thus uses a simple image format. For actual photographing, it is possible to combine parallax component images captured using two cameras arranged in the horizontal direction, as they are. Since some stereoscopic contents are intended for the binocular system, a display apparatus for the parallel-ray II system can desirably deal with a perspective-projection two-image image format, that is, desirably has upward compatibility. To deal with this format, it is possible to interpolate binocular data to display easy-to-see images for the II system. However, it is difficult to interpolate or extrapolate a large number of instances of parallax component images on the basis of a small number of, i.e., two parallax component images for the binocular system. No known methods enable a display apparatus for the parallel-ray II system to display binocular data substantially as it is.
As described above, the conventional three-dimensional image display apparatus for the parallel-ray II system fails to appropriately deal with stereoscopic contents in a variety of image formats, particularly contents obtained through perspective projection.